feat: retain entity relationship evidence in extracted events - #22
feat: retain entity relationship evidence in extracted events#22Pigbibi wants to merge 3 commits into
Conversation
Co-Authored-By: Codex <noreply@openai.com>
🤖 Codex PR Review🚫 Merge blocked: 2 serious issue(s) found in high-risk files ⚖️ Codex Review Arbitration🚫 block: The current findings do not all remain valid. Finding 1 conflicts with the prior blocking contract and is disproved by the current source-of-truth tests: 🚫 Blocking IssuesThese issues must be fixed before this PR can be merged: 1. 🟠 [HIGH] Logic in
|
Co-Authored-By: Codex <noreply@openai.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1d21f8b7b4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if direct_pattern.search(normalized_text): | ||
| relationship = "direct_beneficiary" |
There was a problem hiding this comment.
Prevent generic phrases from becoming direct beneficiaries
When a denylisted alias appears after a direct-beneficiary cue, this branch runs before the generic-alias check below, so text such as funding for cybersecurity is emitted as direct_beneficiary for PANW/CRWD rather than industry_context. That turns sector-level funding language into company-level evidence and can corrupt the relationship fields the commit is adding; the direct pattern should not promote aliases that are only generic context terms.
Useful? React with 👍 / 👎.
| normalized = normalize_match_text(alias).strip() | ||
| if normalized.upper() == alias_record.symbol.upper(): | ||
| return False | ||
| return normalized.casefold() in GENERIC_ALIAS_DENYLIST |
There was a problem hiding this comment.
Normalize configured generic alias variants
This exact membership check misses generic aliases already present in config/core_us_equity_aliases.csv, for example energy related infrastructure alongside the denylisted hyphenated form on config lines 9, 24, 40, and 41. A government-policy item containing the unhyphenated phrase therefore falls through to issuer evidence instead of industry_context; normalize equivalent hyphen/space variants or include the configured variants in the generic classification.
Useful? React with 👍 / 👎.
| for alias_record in aliases: | ||
| evidence = match_evidence(item.text, alias_record, item.source_type) | ||
| if evidence is None: | ||
| continue | ||
| entity_match_type, matched_text = evidence |
There was a problem hiding this comment.
Deduplicate repeated symbols after evidence matching
When an operator-provided aliases CSV has more than one row for the same symbol, this per-alias-record loop appends one event for each row; unlike the previous match_symbols path, nothing deduplicates by symbol before writing. That produces duplicate event_id rows for one source item, which downstream tracker/event-study code counts as separate events and can inflate scores or returns unless the strongest evidence is collapsed per (item_id, symbol).
Useful? React with 👍 / 👎.
| rf"(?:awarded to|contract with|funding for|benefit(?:s|ed)? from)\s+" | ||
| rf"(?:the\s+)?{re.escape(normalized_alias)}", |
There was a problem hiding this comment.
Anchor direct-beneficiary aliases at word boundaries
Because this direct-beneficiary regex inserts the escaped alias without the same trailing boundary used by alias_pattern, an item that mentions a company elsewhere can be upgraded by a prefix match in an unrelated beneficiary, e.g. Ford earnings were discussed. Funding for Fordham University... returns direct_beneficiary for Ford. Reusing the alias boundary here avoids turning longer names that merely start with a ticker/company alias into direct company evidence.
Useful? React with 👍 / 👎.
Co-Authored-By: Codex <noreply@openai.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 21f29cebfb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return None | ||
| relationship, evidence, _ = max( | ||
| matches, | ||
| key=lambda match: (RELATIONSHIP_PRIORITY[match[0]], match[2], len(match[1]), match[1].casefold()), |
There was a problem hiding this comment.
Prefer explicit aliases over generic context
When one item contains both a real company alias and a denylisted context alias for the same symbol, this ordering causes the generic match to win because industry_context outranks unverified; for example, PANW aliases include both Palo Alto Networks and cybersecurity, so a government item like Palo Alto Networks cybersecurity guidance is emitted as industry_context with match_evidence=cybersecurity. That loses the explicit company-name evidence in the new relationship fields, so generic aliases should only win when no non-generic company/ticker alias matched.
Useful? React with 👍 / 👎.
| if not alias_pattern(alias).search(normalized_text): | ||
| continue | ||
| direct_pattern = re.compile( | ||
| rf"(?:awarded to|contract with|funding for|benefit(?:s|ed)? from)\s+" |
There was a problem hiding this comment.
Remove reversed benefits-from cue
When a sentence says another party benefits from a company, this cue treats the company after benefits from as the direct beneficiary. For example, a government item like Domestic suppliers benefit from Intel investment marks INTC as direct_beneficiary, even though the beneficiaries are the suppliers and Intel is just source/context, which overstates company-level support in the new relationship fields.
Useful? React with 👍 / 👎.
| elif source_type == "issuer_release" and is_canonical_name: | ||
| relationship = "issuer" |
There was a problem hiding this comment.
Do not trust third-party names in issuer releases
When any issuer_release item mentions another covered company by its canonical name, this branch promotes that other company to issuer evidence even though source_items.csv has no field identifying which issuer owns the release. For example, an NVIDIA issuer release saying it partnered with Palo Alto Networks would emit PANW as issuer evidence instead of a third-party mention; only promote issuer evidence when the release can be tied to the same symbol.
Useful? React with 👍 / 👎.
|
Superseded after the single allowed remediation. Re-slicing from latest main into a schema/serialization-only contract; matcher behavior will follow in a separate PR. No further changes will be made on this PR. |
Summary
entity_match_type,match_evidence, andrelationship_typeto normalized event schemaindustry_contextwithout changing tracker/event-study scoringScope
Validation
PYTHONPATH=src python3 -m pytest -q tests/test_source_mention_extract.py tests/test_official_event_import.py(14 passed)PYTHONPATH=src python3 -m pytest -q(24 passed)PYTHONPATH=src python3 -m compileall -q src testsgit diff --check